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The study was designed to probe students’ thinking about which numerical values can be 
assigned to algebraic letters. The data from students in Year 7 (n=533), Year 8 (n=377) and 
Year 9 (n=172) was analysed using response patterns. The data confirmed that each year 
contained students with two misconceptions; Different Letter means Different Number and 
the Empty Box misconceptions. The findings provide support for the Steinle et al. (2009) 
hypothesis that a previously identified response pattern is a subset of the Empty Box 
misconception. 

There has been eonsiderable researeh into students’ diffieulties with algebra. Students 
who have an ineomplete or ineorreet interpretation of the meaning of letters, or of the 
equals sign, will have diffieulty making sense of this important topic in mathematics. For 
example, in algebra, letters are used for both u nkn own (speeifie) numbers or as generalised 
numbers (i.e. variables). Christou and Vosniadou (2012) noted that, not only do students 
have a tendeney to substitute only speeifie numbers for letters, they often limit the speeifie 
numbers to natural numbers. 

As well as diffieulties interpreting the letters used in algebra, students ean also have a 
limited interpretation of the equals sign as an instruetion to ‘do’ something (i.e 
operational), rather than as indicating balance (i.e. relational). Asquith, Stephens, Knuth 
and Alibali (2007) eonducted researeh with a small sample of middle sehool teaehers in the 
USA and eoneluded that these teaehers overestimated their students’ ability to give a 
relational definition of the equals sign. 

Lins & Kaput (2004) suggest that the tradition of arithmetic then algebra has 
contributed to students’ diffieulties and reeommend an early introduetion to algebraic 
reasoning (but not to literal symbols). Stephens (2008) investigates student’s transition 
from arithmetie to algebra and the usefulness of students developing relational thinking 
with number sentences to assist with this transition to literal symbols. 

Many researehers have investigated students’ understanding about the use of letters in 
algebra; e.g. MacGregor & Staeey (1997). In eontrast to eareless errors, miseoneeptions 
(also referred to as ‘synthetie conceptions’, by Christou & Vosniadou, 2012) lead to 
predietable errors in student work. Fujii (2003) eategorised students into four groups based 
on their pattern of responses to two items and interviews. Of these four groups, two have 
partieular miseoneeptions and are the foeus of this paper. 

Steinle, Gvozdenko, Priee, Stacey and Pierce (2009) used the term ‘numerieal 
miseoneeptions’ to distinguish these miseoneeptions (involving whieh numerieal values 
ean be assigned to letters) from ‘non-numerieal miseoneeptions’ (sueh as letter as objeet). 
They modified items in Fujii (2003) with slightly different instructions in order to use in an 
online environment (www.smartvie.eom) . Speeifie Mathematics Assessments that Reveal 
Thinking (smart-tests) are designed for teaehers to use for diagnostie purposes and for 
planning future teaehing. These eleetronie tests are quiek for students to eomplete (5-10 
minutes) and the teacher is instantaneously provided with an online diagnosis for each 
student. Aeeompanying the diagnoses are explanations of the assoeiated likely student 
thinking and/or reasons for the errors, along with teaehing suggestions and links to other 
resources. These suggestions are designed to inerease teaehers’ mathematical pedagogical 
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content knowledge for speeifie topics, so affect both short-term teaehing (of eurrent 
students) and long-term teaehing (of future students). 

Steinle et al. (2009) confirmed the students’ misconeeptions highlighted by Fujii 
(2003) about the numerieal values of letters in algebra and found some new response 
patterns which suggest possible new misconceptions. The eurrent study has been designed 
to test their hypothesis that the modification of the Fujii (2003) items for use in an online 
environment has eaused a split within the group of students who have a partieular 
misconception we eall Empty Box. Students with this Empty Box (EB) miseoneeption 
effectively treat the letters in an algebraic equation as if they are boxes, sox + x + x= 12 is 
interpreted as □ + □ + □ = 12, in which case, they believe that there are many correet 
solutions (rather than one), as eaeh box can represent a different value. 

Literature 

Algebra is considered a diffieult part of sehool mathematics mostly due to the need for 
students to develop appropriate interpretations of the abstract symbols involved. Kieran 
(2006) noted that “many coneeptual adjustments are required of the beginning algebra 
student as these signs and symbols shift in meaning from those commonly held in 
arithmetic” (p. 13). 

Christou and Vosniadou (2012) note that if the new information that a student is 
presented with is ineompatible with what is already known, it ereates either “fragmentation 
or systematie miseoneeptions -otherwise kn own as synthetie eoneeptions” (p. 5). Various 
misconceptions are well-known; for example, in the number topic, many students believe 
multiplication makes bigger and division makes smaller (see for example, Lim, 2011). 
Various miseoneeptions within algebra have been reported; for example, the letter as 
object misconception (see Kiiehemann, 1981; MacGregor & Staeey, 1997; Warren, 1998 
and Ak htar & Steinle, submitted) as well as different letters stand for different numbers 
and the same letter does not necessarily stand for the same number (see Eujii, 2003; 
Steinle et al. 2009). 

Kiiehemann (1981) identified that students believed that different letters stand for 
different numbers. For example, over 50% of students (11 to 16 years of age) chose the 
option “never true” when given this equation; L + M+ N= L+ P+ N. The same thinking 
was verified by Booth (1984) who used the same item in a study with 13 to 15 year olds. 
Again, just over 50% ehose “never true”. 

Fujii (2003) studied students’ difficulties with two items involving appropriate choices 
for the numerieal value of letters. Initially he analysed students’ responses to eaeh item 
separately but then he realized that analysing responses aeross both items was more 
revealing. Based on a dditional evidence from interviews, he grouped students into four 
groups; Both problems are correct. Type A, Type B and Others. He defined Type A as 
“different letters stand for different numbers” and Type B as “the same letter does not 
neeessarily stand for the same number” (p. 52). 

Interestingly, Fujii (2003) studied Japanese and American students’ responses and 
noted the samples showed a similar tendency; just over 10% of students in 8* Grade (for 
example) were eorreet on both items. 

Steinle et al. (2009) built onFujii’s work, modifying the items slightly for an online 
environment; see Figure 1. Students were provided with a drop-down box so that they 
eould ehoose the option Right or Wrong on eaeh of the six tasks. 
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Item 498 

Some students had to find some values of x 
to make this equation true: x + x + x = 12 
Mark the work of each student. 

Mary wrote x = 2, .x = 5 and .x = 5 
Millie wrote .x = 9, .x = 2 and .x = 1 
Mandy wrote .x = 4 


Item 502 

Some students had to find some values of .x 
and y to make this equation true: x + y = 16 
Mark the work of each student. 

John wrote .x = 6, y = 10 
Jack wrote .x = 8 and >• = 8 
James wrote .x = 9 and y=l 


Figure 1. Original items used in Steinle et al. (2009). 

Similar to Fuji! (2003), a student’s pattern of responses to the full set of six tasks were 
used to classify students into groups. Steinle et al. (2009) confirmed the same 
misconceptions found by Fujii and used the following labels; Expert (Both problems are 
correct), Different Letter means Different Number (Type A), Empty Box (Type B) and 
Unclassified (Others). 

Table 1 pr ovides the response patterns and accuracy patterns for four groups of 
students. For example, if a student responded to these 6 tasks with this response pattern: 
Wrong, Wrong, Right, Right, Right, Right, then the student was labelled Expert. Table 1 
contains this Expert response pattern and the Expert accuracy pattern (V,V,V,V,V,V), as well 
as number of students who responded in this way (n=23). The final row of the table 
indicates that this pattern occurred four times more than would be predicted if students 
were choosing their answers randomly. 


Table 1 

Response Patterns from Steinle et al. (2009) 


Abbreviated tasks 
within Item 498 
and Item 502 

Expert 

DLDN 

EB 


Newl 

Resp 

Patt 

Acc 

Patt 

Resp 

Patt 

Acc 

Patt 

Resp 

Patt 

Acc 

Patt 

Resp 

Patt 

Acc 

Patt 

Mary x = 2, x = 5,x = 5 

W 

V 

W 

V 

R 

X 

R 

X 

Millie x = 9, x = 2, x = l 

W 

V 

W 

V 

R 

X 

R 

X 

Mandy x = 4 

R 

V 

R 

V 

R 

V 

W 

X 

Johnx = 6, y = 10 

R 

V 

R 

V 

R 

V 

R 

V 

Jack X = 8, y = 8 

R 

V 

W 

X 

R 

V 

R 

V 

James x = 9, y = 7 

R 

V 

R 

V 

R 

V 

R 

V 

Erequency 

23 

70 


74 


29 


Ratio* 

4.0 

12.2 

12.9 

5.1 



Note: Accuracy Pattern (Acc Patt) indicates when a response of R (Right) or W (Wrong) is correct (V). 
* The ratio of observed frequency to expected frequency if all students chose randomly. 


The misconceptions Different Letter means Different Number (DLDN) and Empty Box 
(EB) occurred more than 12 times as often as would be expected if students’ choices were 
random. The pattern labelled Newl was not predicted, and it occurred more often than the 
Expert pattern. Steinle et al. hypothesised that the modification of Eujii’s item might have 
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resulted in the splitting of the EB group on the Mandy task (see row 3 of Table 1). If 
students believe that we need any three numbers which add to 12 in Item 498, then x=4 
(while correct from an Expert’s point of view) is not a suitable answer, and these students 
would then choose Wrong (W) on the Mandy task. 

The focus of this paper is to try to determine whether this hypothesis is reasonable. A 
new version of the instrument was created by including an additional task and the data 
from a new sample of students was analysed. 

Methodology 

The sample of the study was 1082 students from 20 secondary schools in Melbourne; 
Year 7 (n = 533), Year 8 (n = 377) and Year 9 (n = 172). The instrument was a test which 
consisted of two items, from the smart-test algebra module. The test was administered to 
the students by their teachers during their classes in the period January to October 201 1 . As 
the smart-test system is designed to be used by teachers for formative assessment, the 
results of Year 7 students should be interpreted with caution. If teachers are using these 
tests to determine what students know before they teach the topic, then results for Year 7 
could be considered as pre-teaching or baseline data. 

The research instrument (see Eigure 2) was created by adding one new task to the first 
item; “Molly wrote x = 4, x = 4, x = 4”, making a total of seven tasks in these two items. 


Item 2254 (Variation of 498) 

Some students had to find some values ofx 
to make this equation true: x + x + x = 1 2 
Mark the work of each student. 

Mary wrote x = 2, x = 5, x = 5 
Millie wrote x = 9, x = 2, x = 1 
Mandy wrote x = 4 

Molly wrote x = 4, x = 4 and x = 4 


Item 2269 (Same as 502) 

Some students had to find some values ofx 
and y to make this equation true: x + y = 1 6 
Mark the work of each student. 

John wrote x = 6, y = 1 0 
Jack wrote x = 8, y = 8 
James wrote x = 9 and y-1 


Figure 2. Items from updated version of smart-test. 


Results and Discussion 
Responses to six original tasks 

In this section, the results for the new sample (n = 1082) on the six original tasks will 
be compared with the previous study. 

As before, a pattern recognition script was used to search the data for common 
response patterns to the original six tasks. The top four patterns are the same as found 
previously; these are listed in decreasing order in Table 2. As before, DLDN, EB and 
Newl all occur more often than the Expert pattern, and the last row of the table indicates 
that these patterns occur between 6 and 14 times more often than would be predicted if 
students were choosing at random. In this study, Newl occurs more often than EB. These 
patterns account for 692 students (64%) of the total sample (n =1082). 

Hence, we have confirmation that these response patterns do appear in the data and 
Eujii’s interviews provide the evidence that students do ha ve the two particular 
misconceptions he described, and that we labelled DEDN and EB. 
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The high frequency of Newl provides evidence that it was not an anomaly in the 
previous data. Whether this group of students has a new misconception or is part of the EB 
group who have responded differently to the Mandy task is to be determined in the 
following section. 


Table 2 

Frequency of the most common patterns to 6 tasks 


Patterns 

DEDN 

Newl 

EB 

Expert 

Frequency 

232 

187 

157 

116 

Ratio* 

13.7 

11.1 

9.3 

6.9 


* The ratio of observed frequency to expected frequency if all students chose randomly. 


Responses to Molly task 

Steinle et al. (2009) predicted that modifications to Fujii’s items for the online test had 
led to a split of the students with EB thinking on this task: Mandy x=4. Hence, the extra 
task (Molly x = 4, x = 4, x = 4) was introduced in item 2254 to study the behaviour of the 
students who had the response patterns EB and Newl . 

The percentage of each group of students who chose Right and Wrong on the Molly 
task was determined and is shown in Figure 3. Just over 20% of the Experts indicated 
Molly was wrong, while very few of the students in EB and Newl (3% and 5%, 
respectively) chose this response. The similar behaviour of the EB and Newl groups on the 
Molly task, and the fact that these differ from both DEDN and Expert, provides support for 
the hypothesis that EB and Newl students have very similar thinking, and that the different 
format of the Mandy task has split one group into two. 


EB (n = 157) 
New 1 (n = 187) 
DLDN (n = 232) 
Expert (n = 116) 


97% 


3 Vo 


95% 


5‘^o 


85% 


15% 


78% 


22 % 


0% 20% 40% 60% 

□ Molly Right □ Molly Wrong 


80% 


100 % 


Figure 3. Students’ response on Molly task. 


Analysis by year level 

The sample was comprised of 1082 students; Year 7 (n=533). Year 8 (n=377) and Year 
9 (n=172). The prevalence of each group by year level is presented in this section. In this 
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analysis, the Newl and EB groups are eombined and labelled as EB Newl. Any student 
whose response pattern does not match with Expert, DLDN or EB New is labelled as 
Unclassified. Eigure 4 contains the prevalence of the four groups, based on their response 
patterns, by year level. 



■ Expert DDLON □EB_Newl □Unclassified 


Figure 4. Prevalence of four groups based on response patterns, by year level. 

The Year 7 data could be considered to be pre-teaching data; hence it is not a surprise 
that less than 10% of the Year 7 students answered as an Expert on the six tasks of interest. 
That this has increased to 15% in Year 8 is an improvement, but the low level in Year 9 is 
surprising. These are very similar to the results in Steinle et al. (2009) for students in Years 
7 and 8, although there were no Year 9 students in that sample to compare with. This is 
also similar to the results noted by Eujii for the samples in Japan and the USA. 

About 20% to 30% of each year level responded as DLDN and about 30% to 40% of 
each year level responded as EB Newl, which are similar to the results in Steinle et al. 
(2009). About 30% to 45% of each year level did not match any of the other response 
patterns and were labelled as Unclassified; this likewise compares with 30% to 50% of the 
previous study. 


Conclusion 

The focus of the study was to determine if the unexpected Newl response pattern 
found in Steinle et al. (2009) was essentially students who had the Empty Box 
misconception. We have support for this hypothesis; firstly, both groups responded in a 
similar, predicted way to a new task, and secondly, they responded differently to the two 
other groups of students. 

Overall, there is considerable consistency between the results of this study and the 
previous studies. The small percentage of Year 8 and 9 students who answer both items 
correctly (referred to as Experts in this paper) is a concern, and requires more 
investigation. We intend to further develop this smart-test by including parallel items so 
that the diagnosis is based on more responses and hence is more robust. We should also 
add in extra tasks, within each item, where the Empty Box students would choose the 
option wrong-, for example, within Item 2269, where the equation is x +y = 16, “Jill wrote 
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x = l,y = 11”. This would separate those who ehoose Right for eaeh task (perhaps due to 
non-engagement with the tasks) from those with the Empty Box misconception. 

The findings of this study are important for classroom teachers and teacher educators. 
Knowing that there are likely to be many students in each Year 7, 8 and 9 mathematics 
class with the two main misconceptions, Empty Box and Different Letter means Different 
Number, should highlight the need for teachers to make the rules of assigning numerical 
values to letters more explicit. In particular, (i) multiple occurrences of a letter within one 
problem, all have the same value, and (ii) it is possible that two different letters (such as x 
and y) may, in fact, have the same value. In the first case, students are unlikely to make 
sense of the process of transforming equations (for example, x + x + x= 12 can be 
simplified to 3x = 12 and then x = 4) if they believe that the different x’s in the original 
equation can be different numbers (such as 9, 2 and 1). In the second case, students have 
not appreciated that an equation such as x + y = 16 can be used to represent a situation 
where there are two variables and that these do not necessarily have different numerical 
values. For these students, it is almost as if they believe that a graph of x + y = 16 has a 
hole at the point (8,8). 

A major aim of the smart-test system is to increase teachers’ mathematical pedagogical 
content knowledge. We hypothesised that putting data on their own students’ thinking into 
teachers’ hands would make research results come alive for teachers, and hence build their 
capacity. As reported in Steinle & Stacey (2012), there is some promising data from a 
survey of teachers using the system. Of the 127 responses to a multiple choice question ‘As 
a result of using this quiz have you learned something useful for you as a teacher?’, over 
90% indicated that “yes” they did learn something useful; 58 (46%) chose “Yes, very 
valuable learning”. 

As noted by Asquith et al. (2007), while the middle school teachers in their USA 
sample were aware that some students would not have a relational definition of the equals 
sign, the teachers overestimated their students’ ability. The purpose of the smart-test 
system is so that any teacher can obtain data on their students to inform their future 
teaching. 

We conclude with a quote from Steinle et al. (2009, p. 498): 

For approximately 90% of these Year 8 students, then, mathematics lessons containing algebra are 

rendered incomprehensible; these students are trying to learn procedures, without meaning, carried 

out on letters with the wrong meaning. 
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